Communication terminal, communication system, display control method, and recording medium

ABSTRACT

A communication terminal for communicating with a counterpart communication terminal includes: circuitry to control a display to display communication start information reflecting a communication function to be started by the communication terminal, and detect a connection of an external device to the communication terminal to generate a detection result. The external device is to input or output content data to or from the communication terminal to enable the communication terminal to communicate the content data with the counterpart communication terminal using the communication function. When the detection result indicates that the external device is connected to the communication terminal, the circuitry controls the display to display the communication start information in a first display format. When the detection result indicates that the external device is not connected to the communication terminal, the circuitry controls the display to display the communication start information in a second display format.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application is a continuation of U.S. patent applicationSer. No. 15/416,077, filed on Jan. 26, 2017, which is based on andclaims priority pursuant to 35 U.S.C. § 119(a) to Japanese PatentApplication Nos. 2016-015846, filed on Jan. 29, 2016, and 2017-006660,filed on Jan. 18, 2017, in the Japan Patent Office, the entiredisclosure of each is hereby incorporated by reference herein.

BACKGROUND Technical Field

The present invention relates to a communication terminal, acommunication system, a display control method, and a non-transitoryrecording medium.

Description of the Related Art

With the need for reducing costs or times associated with businesstrips, communication systems are widely used, which are capable ofcarrying out videoconferences among remotely located sites through acommunication network such as the Internet. The communication systemsenable transmission or reception of content data among a plurality ofcommunication terminals. The content data may be display data such as animage of a videoconference participant or an image of presentationmaterial, and stroke data reflecting drawings made by the participant.

Japanese Patent Application Publication No. 2015-84211-A disclosessharing of stroke data between electronic whiteboards. Some electronicwhiteboards are capable of exchanging content data other than strokedata with a counterpart electronic whiteboard, for example, using anexternal device that inputs or outputs such content data to beexchanged.

SUMMARY

The inventor of the present invention has recognized that acommunication function of the electronic whiteboard to communicate othercontent data, such as audio data, is not used by some users. In mostcases, the user is not aware of such external device that enables theelectronic whiteboard to communicate other content data. In such case,the communication function of the electronic whiteboard will not beused.

In other cases, even when the user is aware of availability of suchcommunication function, unless the external device is actually providedor connected to the electronic whiteboard, the communication functionwill not be performed by the electronic whiteboard. In such case, theuser can realize that the communication function is not available, onlyafter the user starts communicating with the other user at the other endat least via stroke data, as the communication of other content datafails.

In view of the above, the inventor of the present invention hasrecognized that there is a need for providing information regardingavailability of the communication function on the electronic whiteboard.

Example embodiments of the present invention include a communicationterminal for communicating with a counterpart communication terminal.The communication terminal includes: circuitry to control a display todisplay communication start information reflecting a communicationfunction to be started by the communication terminal, and detect aconnection of an external device to the communication terminal togenerate a detection result. The external device is to input or outputcontent data to or from the communication terminal to enable thecommunication terminal to communicate the content data with thecounterpart communication terminal using the communication function.When the detection result indicates that the external device isconnected to the communication terminal, the circuitry controls thedisplay to display the communication start information in a firstdisplay format. When the detection result indicates that the externaldevice is not connected to the communication terminal, the circuitrycontrols the display to display the communication start information in asecond display format.

Example embodiments of the present invention include a communicationsystem including the communication terminal.

Example embodiments of the present invention include a method ofcontrolling a display of a communication terminal capable ofcommunicating with a counterpart communication terminal. The methodincludes: displaying, on the display of the communication terminal,communication start information reflecting a communication function tobe started by the communication terminal; and detecting a connection ofan external device to the communication, the external device beingconfigured to input or output content data to or from the communicationterminal to enable the communication terminal to communicate the contentdata with the counterpart communication terminal using the communicationfunction. When the detecting indicates that the external device isconnected to the communication terminal, the displaying displays thecommunication start information in a first display format. When thedetecting indicates that the external device is not connected to thecommunication terminal, the displaying displays the communication startinformation in a second display format.

Example embodiments of the present invention include a control programthat controls the communication terminal to perform the above-describeddisplay control method, which is stored in a non-transitory recordingmedium.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

A more complete appreciation of the disclosure and many of the attendantadvantages and features thereof can be readily obtained and understoodfrom the following detailed description with reference to theaccompanying drawings, wherein:

FIG. 1 is a schematic diagram illustrating a communication systemaccording to an example embodiment of the present invention;

FIG. 2 is a schematic block diagram illustrating a hardwareconfiguration of an electronic whiteboard of the communication system ofFIG. 1;

FIG. 3 is a schematic block diagram illustrating a hardwareconfiguration of a communication management system of the communicationsystem of FIG. 1;

FIG. 4 is a schematic block diagram illustrating a hardwareconfiguration of a personal computer (PC), according to an embodiment ofthe present invention;

FIG. 5A is a schematic block diagram illustrating a softwareconfiguration of the electronic whiteboard of FIG. 2, according to anembodiment of the present invention;

FIG. 5B is a schematic block diagram illustrating a softwareconfiguration of the PC of FIG. 4, according to an embodiment of thepresent invention;

FIG. 6 is a schematic block diagram illustrating a functionalconfiguration of the electronic whiteboard, the PC, and thecommunication management system of the communication system of FIG. 1;

FIG. 7 is a schematic block diagram illustrating a drawing controller ofthe electronic whiteboard of FIG. 5A, according to an embodiment of thepresent invention;

FIG. 8 is a data sequence diagram illustrating operation of establishingcommunication among the electronic whiteboards of FIG. 1, according toan embodiment of the present invention;

FIG. 9 is a data sequence diagram illustrating operation of processinglogin, performed by the communication system of FIG. 1, according to anembodiment of the present invention;

FIG. 10 is a data sequence diagram illustrating operation of sharingstroke data, performed by the communication system of FIG. 1, accordingto an embodiment of the present invention;

FIGS. 11A and 11B are an illustration of an example image displayed atthe electronic whiteboard;

FIG. 12 is a flowchart illustrating operation of generating a userinterface image layer, performed by the electronic whiteboard of FIG. 2,according to an embodiment;

FIGS. 13A and 13B are an illustration of example user interface images;

FIG. 14 is a data sequence diagram illustrating operation of startingcommunication, performed by the communication system of FIG. 1,according to an embodiment;

FIG. 15 is a flowchart illustrating operation of displaying acommunication menu, performed by the electronic whiteboard of FIG. 2,according to an embodiment;

FIGS. 16A to 16D are an illustration of example images of acommunication menu, displayed by the electronic whiteboard of FIG. 1;and

FIG. 17A and FIG. 17B are examples of table managed by the electronicwhiteboard of the communication system of FIG. 1.

The accompanying drawings are intended to depict embodiments of thepresent invention and should not be interpreted to limit the scopethereof. The accompanying drawings are not to be considered as drawn toscale unless explicitly noted.

DETAILED DESCRIPTION

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the presentinvention. As used herein, the singular forms “a”, “an” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise.

In describing embodiments illustrated in the drawings, specificterminology is employed for the sake of clarity. However, the disclosureof this specification is not intended to be limited to the specificterminology so selected and it is to be understood that each specificelement includes all technical equivalents that have a similar function,operate in a similar manner, and achieve a similar result.

Referring to the drawings, embodiments of the present invention aredescribed.

<Configuration of Communication System>

Referring to FIG. 1, a configuration of a communication system 1 isdescribed according to an embodiment. FIG. 1 is a schematic diagramillustrating the communication system 1 according to the embodiment.

The communication system 1 includes a plurality of electronic whiteboards 2 a, 2 b, and 2 c, a personal computer (PC) 8, and acommunication management system 50.

The electronic whiteboards 2 a, 2 b, and 2 c are connected via a firstcommunication network 9 a to be communicable with one another. Theelectronic whiteboards 2 a, 2 b, and 2 c are respectively provided withdisplays 3 a, 3 b, and 3 c.

The electronic whiteboards 2 a, 2 b, and 2 c respectively cause thedisplays 3 a, 3 b, and 3 c to display drawing images, drawn withelectronic pens 4 a, 4 b, and 4 c or the user's hands Ha, Hb, and Hc.For simplicity, any arbitrary one of the electronic whiteboards 2 a, 2b, and 2 c is referred to as the electronic whiteboard 2. Similarly, anyarbitrary one of the displays 3 a, 3 b, and 3 c is referred to as thedisplay 3. Any arbitrary one of the electronic pens 4 a, 4 b, and 4 c isreferred to as the electronic pen 4. Any arbitrary one of the user'shands Ha, Hb, and Hc is referred to as the hand H. Further, theelectronic whiteboard 2 may change the drawing image being displayed onthe display 3, according to an event made by the user's hand H. Forexample, in response to the user's gesture indicating size enlargement,size reduction, or turning pages, the electronic whiteboard 2 changesthe image size or switches a screen.

The electronic whiteboards 2 a, 2 b, and 2 c are respectivelyconnectable to the universal serial bus (USB) memories 5 a, 5 b, and 5c. The electronic whiteboards 2 a, 2 b, and 2 c may read or writeelectronic files, which may be written in portable document format(PDF), onto or from the USB memories 5 a, 5 b, and 5 c. The electronicwhiteboards 2 a, 2 b, and 2 c are respectively connected to note PCs 6a, 6 b, and 6 c, via cables, to be communicable in compliance with thestandard such as DisplayPort, Digital Visual Interface (DVI),High-Definition Multimedia Interface (HDMI; Registered Trademark), orVideo Graphics Array (VGA). Any one of the USB memories 5 a, 5 b, and 5c is referred to as the USB memory 5. Any one of the note PCs 6 a, 6 b,and 6 c is referred to as the note PC 6.

The stroke data, which is the drawing drawn on the display 3 of theelectronic whiteboard 2, captured at a first site is transmitted to asecond site through the first communication network 9 a for outputthrough the display 3 of the electronic whiteboard 2 at the second site.Similarly, the stroke data, which is the drawing drawn on the display 3of the electronic whiteboard 2 at the second site, is transmitted to thefirst site through the first communication network 9 a for outputthrough the display 3 of the electronic whiteboard 2 at the first site.As described above, the drawing images can be shared among remotelylocated sites, such that the communication system 1 with the electronicwhiteboards 2 can facilitate communication in a videoconference.

In addition to the electronic whiteboard 2, the drawings, such as strokedata, can be shared by any desired communication terminal connected tothe first communication network 9 a. Examples of such communicationterminal include, PC, videoconference terminal, tablet, smart phone,digital signage, telestrator, image processing apparatus, portableinformation processing device, digital video camera, digital stillcamera, and game machine. The telestrator may be used for providinginformation on sports or weather. The image processing apparatus may beused to provide a remote medical diagnosis system. The game machine maybe one example of any terminal capable of providing a frame of imagedata.

The electronic whiteboard 2 c, the PC 8, and the management system 50are connected via a second communication network 9 b to be communicablewith one another.

The electronic whiteboard 2 c and the PC 8 communicate with each otherto transmit or receive image data and audio data for videoconferencing,as an example of content data. The display data is any image data thatcan be displayed on a display, such as an image captured at one siteduring videoconference. More specifically, the display data may be avideo image or a still image, or both of the video image and the stillimage. For simplicity, the image data and the display data may be usedinterchangeably.

The management system 50 centrally manages communicates states of theelectronic whiteboard 2 c and the PC 8.

While the electronic whiteboard 2 c and the PC 8 are connected to thesecond communication network 9 b in this example, any othercommunication terminal may be connected to the second communicationnetwork 9 b to share image data with the electronic whiteboard 2 c andthe PC 8. Examples of such communication terminal include, but notlimited to, videoconference terminal, tablet, smart phone, digitalsignage, telestrator, image processing apparatus, portable informationprocessing terminal, digital video camera, digital still camera, andgame machine.

In this embodiment, the electronic whiteboard 2 c activates a module forsharing stroke data, and a module for sharing image data, to sharestroke data, image data, and audio data among the communicationterminals.

In one example, the first communication network 9 a is the Intranet, andthe second communication network 9 b is the Internet. The communicationnetworks 9 a and 9 b may be previously determined according tocommunications protocol of the communication terminals in thecommunication system 1, and may be selected from the Internet, localarea network (LAN), the Intranet, and mobile phone network. Thecommunication networks 9 a and 9 b may be the same communicationnetwork, such as the Internet.

The example illustrated in FIG. 1 uses the USB memory 5 as a memoryattachable to the electronic whiteboard 2, however, any desired memorymay be used such as a SD card.

<Hardware Configuration of Electronic Whiteboard>

FIG. 2 illustrates a hardware configuration of the electronic whiteboard2, according to the embodiment. FIG. 2 is a schematic block diagramillustrating a hardware configuration of the electronic whiteboard 2illustrated in FIG. 1.

As illustrated in FIG. 2, the electronic whiteboard 2 includes a centralprocessing unit (CPU) 201 that controls entire operation of theelectronic whiteboard 2, a read only memory (ROM) 202 that stores aprogram for operating the CPU 201 such as an initial program loader(IPL), a random access memory (RAM) 203 that operates as a work area forthe CPU 201, a solid state drive (SSD) 204 that controls reading orwriting of various types of data including control program stored in amemory, a network controller 205 that controls communication via thecommunication network 9 a or 9 b, and an external memory controller 206that controls communication with the USB memory 5. The electronicwhiteboard 2 further includes a camera 207 that captures an object undercontrol of the CPU 201, an imaging element I/F 208 that controlsoperation of the camera 207, a capture device 211 that transmits imagedata (still or moving image) for display at the note PC 6, a graphicsprocessing unit (GPU) 212 that processing graphics, and a displaycontroller 213 that controls transmission of image data output from theGPU 212 to the display 3. The electronic whiteboard 2 further includes asensor controller 214 and a contact sensor 215. The sensor controller214 controls operation of the contact sensor 215. The contact sensor 715detects a touch onto the display 3 with the electronic pen 4 or theuser's hand H. In this example, the contact sensor 215 senses a touchinput to a specific coordinate on the display 3 using the infraredblocking system. More specifically, the display 3 is provided with twolight receiving elements disposed on both upper side ends of the display3, and a reflector frame. The light receiving elements emit a pluralityof infrared rays in parallel to a touch panel of the display 3. Thelight receiving elements receive lights passing in the direction that isthe same as an optical path of the emitted infrared rays, which arereflected by the reflector frame. The contact sensor 215 outputs anidentifier (ID) of the infrared ray that is blocked by an object (suchas the user's hand) after being emitted from the light receivingelements, to the sensor controller 214. Based on the ID of the infraredray, the sensor controller 214 detects a specific coordinate that istouched.

The contact sensor 215 is not limited to the infrared blocking systemtype, and may be a different type of detector, such as a capacitancetouch panel that identifies the contact position by detecting a changein capacitance, a resistance film touch panel that identifies thecontact position by detecting a change in voltage of two opposedresistance films, or an electromagnetic induction touch panel thatidentifies the contact position by detecting electromagnetic inductioncaused by contact of an object to a display. Alternatively, the contactsensor 215 may identify the contact position using the camera 207.

The electronic whiteboard 2 further includes an electronic pencontroller 216. The electronic pen controller 216 communicates with theelectronic pen 4 to detect a touch by the tip or bottom of the pen 4 tothe display 3. In addition or in alternative to detecting a touch by thetip or bottom of the pen 4, the electronic pen controller 216 may alsodetect a touch by another part of the electronic pen 4, such as a partheld by a hand.

The electronic whiteboard 2 further includes an audio input/output I/F224 that processes input or output of audio signals between an externalmicrophone 222 and an external speaker 223 under control of the CPU 201,operation key(s) 225 that receives user inputs, and a power switch 226that turns on or off the electronic power of the electronic whiteboard2. The microphone 222 that inputs audio and the speaker 223 that outputsaudio may each be incorporated in the electronic whiteboard 2.

The electronic whiteboard 2 further includes a bus line 230, such as anaddress bus or a data bus, which electrically connects the elementsillustrated in FIG. 2.

The control program for the electronic whiteboard 2 may be stored on acomputer-readable recording medium such as a CD-ROM for distribution, orstored on a server on any desired network to be downloaded.

<Hardware Configuration of Communication Management System>

Referring to FIG. 3, a hardware configuration of the communicationmanagement system 50 is described according to an embodiment of thepresent invention. FIG. 3 is a schematic block diagram illustrating ahardware configuration of the communication management system 50according to the embodiment.

The management system 50 includes a CPU 501 that controls entireoperation of the management system 50, a ROM 502 that stores acommunication control program, a RAM 503 that operates as a work areafor the CPU 501, a hard disk (HD) 504 that stores various data, a HDD505 that controls reading or writing of various data from or to the HD504 under control of the CPU 501, and a medium I/F 507 that controlsreading or writing of data with respect to a recording medium 506 suchas a flash memory. The management system 50 further includes a displayI/F 508 that displays various information on a display 516 such as acurser, menu, window, character, or image, a network I/F 509 thatcontrols transmission of data through the communication network 9 a or 9b, a keyboard 511 provided with a plurality of keys for inputting acharacter, numeral, or instruction, and a mouse 512 that selects orexecutes an instruction through selection of a processing target ormovement of a mouse cursor. The management system 50 further includes aCD-ROM drive 514 that controls reading or writing with respect to aCD-ROM 513 as an example of removable memory, an external device I/F 515that controls transmission of various data with an external device, anda bus line 510, such as an address bus or a data bus, that electricallyconnects the elements illustrated in FIG. 3.

Note that the communication control program may be recorded in a file ina format installable or executable on a computer-readable recordingmedium such as the recording medium 506 or the CD-ROM 513 fordistribution. The communication control program may be stored on the HD504.

<Hardware Configuration of PC>

FIG. 4 illustrates a hardware configuration of the PC 8, as an exampleof videoconference terminal, according to the embodiment. The PC 8includes a central processing unit (CPU) 801, a read only memory (ROM)802, a random access memory (RAM) 803, a flash memory 804, a solid statedrive (SSD) 805, a medium interface (I/F) 807, a power switch 809, a busline 810, a network I/F 811, a camera 812, an imaging element 1/F 813, amicrophone 814, a speaker 815, an audio input/output I/F 816, a displayI/F 817, an external device I/F 818, a keyboard 821, and a mouse 822.The CPU 801 controls entire operation of the PC 8. The ROM 802 stores acontrol program for operating the CPU 801 such as an Initial ProgramLoader (IPL). The RAM 803 is used as a work area for the CPU 801. Theflash memory 804 stores various data such as a communication controlprogram, image data, and audio data. The SSD 805 controls reading orwriting of various data with respect to the flash memory 804 undercontrol of the CPU 801. In alternative to the SSD, a hard disk drive(HDD) may be used. The medium I/F 807 controls reading or writing ofdata with respect to a recording medium 806 such as a flash memory.

The network I/F 811 controls communication of data with an externaldevice through the second communication network 9 b. The camera 812 isan example of imaging device capable of capturing an object undercontrol of the CPU 801, and is incorporated in the PC 8. The imagingelement 1/F 813 is a circuit that controls driving of the camera 812.The microphone 814 is an example of audio collecting device capable ofinputting audio under control of the CPU 801, and is incorporated in thePC 8. The audio I/O I/F 816 is a circuit for inputting or outputting anaudio signal between the microphone 814 and the speaker 815 undercontrol of the CPU 801. The display I/F 817 is a circuit fortransmitting display data to the external display 820 under control ofthe CPU 801. The external device I/F 818 is an interface circuit thatconnects the PC 8 to various external devices. The keyboard 821 is oneexample of input device provided with a plurality of keys for allowing auser to input characters, numerals, or various instructions. The mouse812 is one example of input device for allowing the user to select aspecific instruction or execution, select a target for processing, ormove a curser being displayed.

The bus line 810 is an address bus or a data bus, which electricallyconnects the elements in FIG. 4 such as the CPU 801.

The display 820 may be a liquid crystal or organic electroluminescence(EL) display that displays an image of a subject, an operation icon, orthe like. The display 820 is connected to the display I/F 817 by a cable820 c. The cable 820 c may be an analog red green blue (RGB) (videographic array (VGA)) signal cable, a component video cable, ahigh-definition multimedia interface (HDMI) signal cable, or a digitalvideo interactive (DVI) signal cable.

The camera 812 includes a lens and a solid-state imaging element thatconverts an image (video) of a subject to electronic data by convertinglight to electric charge. As the solid-state imaging element, forexample, a complementary metal-oxide-semiconductor (CMOS) or acharge-coupled device (CCD) is used. The external device I/F 818 iscapable of connecting an external device such as an external camera, anexternal microphone, or an external speaker through a Universal SerialBus (USB) cable or the like. In the case where an external camera isconnected, the external camera is driven in preference to the built-incamera 812 under control of the CPU 801. Similarly, in the case where anexternal microphone is connected or an external speaker is connected,the external microphone or the external speaker is driven in preferenceto the built-in microphone 814 or the built-in speaker 815 under controlof the CPU 801.

The recording medium 806 is removable from the PC 8. The recordingmedium 801 can be any non-volatile memory that reads or writes dataunder control of the CPU 801, such that any memory such as anelectrically erasable and programmable read-only memory (EEPROM) may beused instead of the flash memory 804.

<Software Configuration>

FIG. 5A is a schematic block diagram illustrating a softwareconfiguration of the electronic whiteboard 2 of FIG. 2, according to anembodiment of the present invention. As illustrated in FIG. 5A, theelectronic whiteboard 2 is installed with an operating system (OS) 2020,first communication application A11, and second communicationapplication B10, each of which operates on a work area 2010 of the RAM203.

The OS 2020 is basic software that controls entire operation of theelectronic whiteboard 2 through providing basic functions. Thecommunication application A11 and B10 each enable the electronicwhiteboard 2 to communicate with the other electronic whiteboard (orcommunication terminal), using different communication protocols. Thefirst communication application A11 is client application, whichprovides the electronic whiteboard 2 with a communication controlfunction of controlling transmission of stroke data with the othercommunication terminal, and an image processing function of outputtingstroke data as an image. The second communication application B10 isclient application, which provides the electronic whiteboard 2 with acommunication control function of controlling transmission of image dataand audio data (such as image data and audio data for videoconferencing)with the other communication terminal, an image processing function ofinputting or outputting image data, and an audio processing function ofinputting or outputting audio data.

The OS 2020 and the first communication application A11 are installedonto the electronic whiteboard 2 before shipment. The secondcommunication application B10 may be installed onto the electronicwhiteboard 2, after shipment. In the following, it is assumed that theelectronic whiteboard 2 c is installed with the second communicationapplication B10, and the electronic whiteboards 2 a and 2 b are notinstalled with the second communication application B10.

FIG. 5B is a schematic block diagram illustrating a softwareconfiguration of the PC 8 of FIG. 4, according to an embodiment of thepresent invention. As illustrated in FIG. 5B, the PC 8 is installed withoperating system (OS) 8020, and the third communication application B11,which may be deployed on a working area 8010 of the RAM 803.

The OS 8020 is basic software that controls entire operation of the PC 8through providing basic functions. The third communication applicationB11 provides the PC 8 with the communication control function ofcontrolling transmission of image data with the other communicationterminal, and the image processing function of outputting image data asan image.

The OS 8020 and the third communication application B11 are installedonto the PC 8 before or after shipment.

<Functional Configuration of Communication System>

Now, a functional configuration of the communication system 1 isdescribed according to an embodiment of the present invention. FIG. 6 isa schematic block diagram illustrating a functional configuration of theelectronic whiteboard 2, the PC 8, and the management system 50 of thecommunication system of FIG. 1. In FIG. 6, the electronic whiteboards 2a, 2 b, and 2 c are connected through the first communication network 9a to transmit or receive data. Further, in FIG. 6, the electronicwhiteboard 2 c, the PC 8, and the management system 50 are connectedthrough the second communication network 9 b to transmit or receivedata.

The electronic whiteboard 2 includes hardware of FIG. 2, which operatesin cooperation with the control program, to implement the functionalconfiguration of FIG. 6. The electronic whiteboard 2 includes a memory2000 implemented by the SSD 204. The PC 8 includes the hardware of FIG.4, which operates in cooperation with the control program, to implementthe functional configuration of FIG. 6. The PC 8 includes a memory 8000implemented by the SSD 805.

The electronic whiteboard 2 may function as a “host terminal” thatrequests to start communication to share images, or a “participantterminal” that participates in communication started by the hostterminal. The electronic whiteboard 2 mainly includes a client section(“client”) 20 and a server section (“server”) 90. The client 20 and theserver 90 are functions performed by one electronic whiteboard 2, withactivation of the first communication application A11. Alternatively,the electronic whiteboard two may only function as the client 20, whilecausing other apparatus such as a distribution control apparatus tooperate as the server 90.

In case the electronic whiteboard 2 operates as the host terminal, suchelectronic whiteboard 2 implements both of the client 20 and the server90. In case the electronic whiteboard 2 operates as the participantterminal, such electronic whiteboard 2 implements the client 20 but notthe server 90. For example, in case the electronic whiteboard 2 aoperates as the host terminal, and the electronic whiteboards 2 b and 2c each operate as the participant terminal, the client 20 of theelectronic whiteboard 2 a communicates with the client 20 of each of theelectronic whiteboards 2 b and 2 c, through the server 90 in theelectronic whiteboard 2 a. The client 20 of each one of the electronicwhiteboards 2 b and 2 c communicates with the client 20 of theelectronic whiteboard 2 a, 2 b, or 2 c, through the server 90 in theelectronic whiteboard 2 a.

The electronic whiteboard 2 b and the PC 8 implement the client 80, withactivation of the second communication application B10 and the thirdcommunication application B11, respectively. The client 80 communicateswith the client 80 of the other communication terminal through acommunication session established based on a call control by themanagement system 50 operating as a server.

The client 20 starts operating with activation of the firstcommunication application A11 on the electronic whiteboard 2. The client20 includes a communication controller 21, a drawing controller 22, anda detector 23.

The communication controller 21, which may be implemented by theinstructions of the CPU 201, or the network controller 205, controlscommunication with the other electronic whiteboard 2, or the server 90or the client 80 in the local electronic whiteboard 2.

The drawing controller 22, which may be implemented by the instructionsof the CPU 201, performs image processing on stroke data that isgenerated in response to operation on the display 3, or various dataobtained from any desired device. Examples of such device include, butnot limited to, the USB memory 5, the note PC 6, the communicationcontroller 21, and the memory 2000. The drawing controller 22 generatesan image layer based on the processed image data, and controls output ofa combined image on which a plurality of image layers are superimposedone above the other.

The detector 23, which may be implemented by the instructions of the CPU201, detects a connection between an audio device and the electronicwhiteboard 2, based on a determination of whether the OS 2020 detectsthe microphone 222 or the speaker 223.

FIG. 7 illustrates a functional configuration of the drawing controller22 according to an embodiment. The drawing controller 22 includes astroke processor 22A, an obtainer 22B, a converter 22X, a combiner 22Y,and a page processor 22Z.

The stroke processor 22A, which may be implemented by the instructionsof the CPU 201, generates stroke data that reflects the user's drawingon the display 3 with the electronic pen 4 or the user's hand H. Morespecifically, the stroke data may be a stroke of a character, mark, orany other drawing, which may be drawn by the user. As described belowreferring to tables 1 to 4, in this disclosure, the stroke correspondsto a drawing made by a user with a single unbroken movement, forexample, with the electronic pen.

The obtainer 22B, which may be implemented by the instructions of theCPU 201, obtains data stored in the memory 2000.

The converter 22X, which may be implemented by the instructions of theCPU 201, converts various data. For example, the converter 22X performstextization, real (binary) data conversion, serialization,deserialization, encoding, and decoding.

The combiner 22Y, which may be implemented by the instructions of theCPU 201, generates an image layer based on each type of data, and causethose image layers to be superimposed one above the other. Examples ofimage data types include a user interface (UI) data, stroke data, mediumdata (display data), background image data, and object data. The UI datais used for generating a user interface, which includes, for example, amessage for display to the user and an icon for selection by the user.The background image data is used for generating a background image,such as a frame to be used for defining a layout of the image. Thestroked data, the medium data, and the object data are described belowin detail.

The page processor 22Z, which may be implemented by the instructions ofthe CPU 201, associates the stroke data and the display data for thesame one page, and stores this set of stroke data and display data in apage data memory 220 of the memory 2000.

The page data memory 220 is a memory area of the memory 2000, whichstores page data as illustrated in Table 1. Table 1 illustrates anexample data structure of page data. The page data includes one-pagedata to be displayed on the display 2, which includes stroke arrangementdata (items of stroke data for one character, etc.), and medium data asan example of display data.

TABLE 1 STROKE PAGE DATA ARRANGEMENT MEDIUM ID START TIME END TIME DATAID DATA ID S001 20150522152034 20150522152402 st001 m001 S00220150522152415 2015522152825 st002 m002 S003 2015052215334520150522154033 st003 m003 . . . . . . . . . . . . . . .

The page data includes a page data ID for identifying a specific page,start time when display of that page is started, end time when drawingto that page by a stroke drawing or a gesture is stopped, a strokearrangement data ID for identifying stroke arrangement data, and amedium data ID for identifying medium data. The stroke arrangement datais to be used for displaying a set of stroke drawings on the display 3,as the stroke drawings are made by the user with the electronic pen 4 orthe user's hand H. The medium data is to be used for displaying theother image (referred to as the display data) on the display 3, togetherwith the stroke drawing.

The stroke arrangement data includes various information as illustratedin Table 2 (FIG. 17A). Table 2 illustrates an example data structure ofstroke arrangement data. As illustrated in Table 2, one strokearrangement data, identified with the stroke arrangement data ID inTable 1, includes a plurality of items of stroke data, eachcorresponding to one stroke drawing. Each stroke data includes a strokedata ID for identifying that stroke data, start time when drawing ofthat stroke starts, end time when drawing of that stroke ends, a colorof the stroke, a width of the stroke, and a coordinate arrangement dataID for identifying arrangement of points of the stroke. For example, incase the user draws the alphabet “S” with the electronic pen 4 in onestroke, one stroke data ID is assigned to that stroke drawing “S”. Incase the user draws the alphabet “T” with the electronic pen 4 in twostrokes, two stroke data IDs are assigned to those two stroke drawings“T”.

The coordinate arrangement data includes various information asillustrated in Table 3 (FIG. 17B). Table 3 illustrates an example datastructure of the coordinate arrangement data. The coordinate arrangementdata includes a single point (X coordinate value, Y coordinate value) onthe display 3, difference time (ms) indicating a difference between thetime when that point is drawn and the start time when drawing of thestroke starts, and pressure by the electronic pen 4 on that singlepoint. That is, one item of coordinate arrangement data in Table 2 is acollection of single points in Table 3. For example, in case the userdraws the alphabet “S” with the electronic pen 4 in one stroke, aplurality of points will be drawn, such that the coordinate arrangementdata assigned with a specific data ID (such as, c001) corresponds tothose points in the stroke drawing that are managed with Table 3.

The medium data of the page data in Table 1 includes various informationas illustrated in Table 4. Table 4 illustrates an example data structureof medium data. As illustrated in Table 4, the medium data includes amedium data ID for identifying the medium data as managed with Table 1,a type of the medium data, recording time when the medium data isrecorded, a position of an image to be displayed on the display 3 basedon the medium data (X coordinate, Y coordinate), a size of the image(width, height), and data indicating content of the medium data. Theposition of the image to be displayed based on the medium data isdefined by the upper left corner of that image, when the X and Ycoordinates of the upper left corner of the display 3 are set to (0, 0).

TABLE 4 MEDIUM DATA RECORDING DATA ID TYPE TIME X Y WIDTH HEIGHT DATAm001 IMAGE 20150522152632 1400 50 400 300 abc.jpg . . . . . . . . . . .. . . . . . . . . . . . .

Now, a functional configuration of the server 90 is described. Theserver 90 includes a communication controller 91.

The communication controller 91, which may be implemented by theinstructions of the CPU 201, controls communication with thecommunication controller 21 of the client 20. As described above, theclient 20 may be the client 20 in the same electronic whiteboard 2, orthe client 20 in the external electronic whiteboard 2 that iscommunicable via the first communication network 9 a.

The client 80 of the electronic whiteboard 2 starts operating withactivation of the second communication application B10 on the electronicwhiteboard 2. The client 80 of the PC 8 starts operating with activationof the third communication application B11 on the PC 8. The client 80includes a communication controller 81 and a display controller 82.

The communication controller 81 of the client 80, which may beimplemented by the instructions of the CPU 201 or 801, and the networkcontroller 205 or the network I/F 811, controls communication with thecommunication controller 81 of the client 80 in the external electronicwhiteboard 2, or the communication controller 51 of the managementsystem 50. The communication controller 81 of the electronic whiteboard2, which may be implemented by the instructions of the CPU 201, controlscommunication with the communication controller 21 of the client 20.

The display controller 82 of the client 80 is implemented by theinstructions from the CPU 801, which is generated with activation of thecommunication application B11. The display controller 82 performs imageprocessing based on data obtained from the communication controller 21or the memory 8000, and controls output of processed image data.

The management system 50 includes the hardware of FIG. 3, which operatesin cooperation with the control program, to implement the functionalconfiguration of FIG. 6. The management system 50 further includes amemory 5000, which may be implemented by the HD 504.

The communication controller 51 of the management system 50, which maybe implemented by the instructions of the CPU 501, controlscommunication with the communication controller 81 of the client 80 inthe electronic whiteboard 2 or the PC 8.

<Operation>

Now, operation of controlling display of various data is describedaccording to an embodiment.

Referring to FIG. 8, operation of establishing communication among theelectronic whiteboards 2 a, 2 b, and 2 c, with the communicationapplication A11, is described according to the embodiment. As the powerswitch 226 of the electronic whiteboard 2 a is turned on by the user,the first communication application A11 is activated to start processingby the client of the electronic whiteboard 2 a (S1). In response toreceiving a request for starting a conference through operation onto thedisplay 3 a, the communication controller 21 of the client 20 transmitsan instruction to start processing by the server 90, to thecommunication controller 91 of the server 90 in the same electronicwhiteboard 2 a. Accordingly, the electronic whiteboard 2 a is able tostart various processing by the server 90, in addition to processing bythe client 20 (S2).

The communication controller 91 of the electronic whiteboard 2 agenerates connection data to be used for establishing connection withthe electronic whiteboard 2 a. The client 20 causes the display 3 a todisplay the generated connection data (S3). The connection data includesthe IP address of the host terminal, and a pass code that is generatedfor the communication session to be established for image sharing. Thepass code, which is generated, is stored in the memory 2000. The user atthe electronic whiteboard 2 a, who now knows the connection data,informs the other users at the electronic whiteboards 2 b and 2 c ofthat connection data through telephone or electronic mail.

As the user at the electronic whiteboard 2 b turns on the power switch226, the communication application A11 is activated to start processingof the client 20 in the electronic whiteboard 2 b (S4). Similarly, asthe user at the electronic whiteboard 2 c turns on the power switch 226,the communication application A11 is activated to start processing ofthe client 20 in the electronic whiteboard 2 c (S5). In response to auser input to the display 3 b that requests connection, thecommunication controller 21 of the client 20 in the electronicwhiteboard 2 b transmits a participation request, with the pass code, tothe communication controller 91 of the server 90 in the electronicwhiteboard 2 a (S6). Similarly, the electronic whiteboard 2 c transmitsa participation request, with the pass code, to the electronicwhiteboard 2 a (S7). The communication controller 91 of the electronicwhiteboard 2 a receives the participation requests (with the pass code),respectively, from the electronic whiteboards 2 b and 2 c.

Next, the communication controller 91 authenticates the electronicwhiteboards 2 b and 2 c, based on a match between the pass codes thatare received from the electronic whiteboards 2 b and 2 c, and the passcode stored in the memory 2000 (S8). The communication controller 91transmits authentication results to the clients 90 of the electronicwhiteboards 2 b and 2 c (S9, S10). When the authentication resultindicates that the electronic whiteboards 2 b and 2 c are authenticated,the electronic whiteboard 2 a that operates as the host terminalestablishes a communication, respectively, with the electronicwhiteboard 2 b and electronic whiteboard 2 c that operate as participantterminals (S11, S12). In the following, it is assumed that content datatransmitted among the electronic whiteboards 2 a, 2 b, and 2 c includestroke data.

Referring to FIG. 9, operation of processing login by the electronicwhiteboard 2 c and the PC 8, to start sharing of image data and audiodata is described according to an embodiment.

In response to a user input to the display 3 c that requests activation,the client 20 of the electronic whiteboard 2 c activates the secondcommunication application B10 to start processing by the client 80(S21).

The communication controller 81 of the client 80 transmits a loginrequest to the management system 50 (S22). The login request includes,for example, a user ID and a password, as account information. Thecommunication controller 51 of the management system 50 receives thelogin request.

The management system 50 authenticates the electronic whiteboard 2 cthat requests for log in (S23). More specifically, the management system50 determines whether a pair of the user ID and the password in thelogin request matches any pair of the user ID and the password in thememory 5000. The communication controller 51 of the management system 50transmits an authentication result to the electronic whiteboard 2 c(S24). The communication controller 81 of the electronic whiteboard 2 creceives the authentication result. In the following, it is assumed thatthe authentication result indicates that the electronic whiteboard 2 cis allowed for log in.

The memory 2000 of the electronic whiteboard 2 c has a storage area tostore state information indicating a communication state of the client80. According to the received authentication result indicating thatauthentication is successful, the communication controller 81 of theelectronic whiteboard 2 c stores in the memory 2000 the stateinformation “online” (S25).

As the PC 8 receives a request for activating the third communicationapplication B11, by a user input through the keyboard 821 or the mouse822, the third communication application B11 is activated to startprocessing by the client 80 of the PC 8 (S31).

The communication controller 81 of the client 80 transmits a loginrequest to the management system 50 (S32). The management system 50receives the login request.

The management system 50 authenticates the PC 8 that requests for log in(S33). The management system 50 transmits an authentication result tothe PC 8 (S34). In the following, it is assumed that the authenticationresult indicates that the PC 8 is allowed for log in.

Referring to FIG. 10, operation of sharing stroke data among theplurality of electronic whiteboards 2 a, 2 b, and 2 c is describedaccording to an embodiment.

As the user at the electronic whiteboard 2 c draws a stroke image on theelectronic whiteboard 2 c with the electronic pen 4 c, the strokeprocessor 22A generates stroke data based on the coordinate indicatingthe contact portion on the display 3 c with the electronic pen 4 c. Thestroke data includes a plurality of parameters as described referring toTables 1 to 3. The converter 22X groups a unit of stroke data reflectinga plurality of stroke drawings, and serializes the group of stroke data(S62). The communication controller 21 of the electronic whiteboard 2 ctransmits the serialized stroke data to the communication controller 91of the electronic whiteboard 2 a that is the host terminal (S63). Theelectronic whiteboard 2 c may transmit stroke data, one stroke drawingby one stroke drawing, to the electronic whiteboard 2 a as the hostterminal, as the stroke drawing is detected.

As the stroked data is received from the electronic whiteboard 2 c, thecommunication controller 91 of the electronic whiteboard 2 a transmitsthe received stroke data to the client 20 of the electronic whiteboard 2a. The communication controller 21 of the client 20 in the electronicwhiteboard 2 a then receives the transmitted stroke data. The converter22X of the drawing controller 22 of the client 20 de-serializes thestroke data (S64). The page processor 22Z stores the de-serializedstroke data in the page data memory 220, as newly obtained stroke datato be added to previously stored stroke data. The combiner 22Y furthergenerates a stroke image, that is, a group of strokes, based on thestroke data stored in the page data memory 220. The combiner 22Ygenerates an output image, by combining the items of stroke data storedin the page data memory 220.

The combiner 22Y combines the generated output image, the generatedstroke image, a user interface (UI) image, and a background image storedin the page data memory 220, to generate a combined image having fourlayers superimposed one above the other. The UI image may be previouslystored in the page data memory 220. The combiner 22Y transmits thecombined image to the display 3 a for display at the display 3 a (S65).The combined image may be an image having a plurality of image layersbeing superimposed one above the other as illustrated in FIG. 11A, whichmay be displayed as one image as illustrated in FIG. 11B. FIG. 11Aillustrates an example combined image 140 having the UI image layer 140a, stroke image layer 140 b, output image layer 140 c, and backgroundimage layer 140 d, which are superimposed one above the other. FIG. 11Billustrates the combined image 140 when displayed at the display 3.

Referring back to FIG. 10, the communication controller 91 of theelectronic whiteboard 2 a transmits the serialized stroke data,transmitted from the electronic whiteboard 2 c, to the clients 20 of theelectronic whiteboards 2 b and 2 c, respectively (S66, S67). Thecommunication controller 21 of the client 20 in each one of theelectronic whiteboards 2 b and 2 c receives the transmitted stroke data.The client 20 in each of the electronic whiteboards 2 b and 2 c causeseach one of the displays 3 b and 3 c to display the combined image,which includes the output image generated based on the stroke datatransmitted from the electronic whiteboard 2 a. Since this operation issubstantially similar to the operation described above, referring to S64to S66 for the case of the client 20 of the electronic whiteboard 2 a,description thereof is omitted (S68 to S71).

Referring to FIG. 12, operation of generating the UI image layer,performed by the combiner 22Y of the electronic whiteboard 2 c at S71,is described according to an embodiment. The following describes anexample operation to be performed by the electronic whiteboard 2 c, butany other electronic whiteboard 2 is able to perform operation ofgenerating the UI image layer in a substantially similar manner.

The page data memory 200 of the electronic whiteboard 2 c previouslystores image data of a plurality of types of communication icons to beused for videoconferencing, which are to be displayed with differentbrightness values. In this example, the icons include a firstcommunication icon to be displayed with a predetermined brightnessvalue, and a second communication icon to be displayed with half a valueof the predetermined brightness value. For simplicity, the firstcommunication icon and the second communication icon may be referred toas the brighter icon and the less bright icon. When generating the UIimage, the first communication icon is made selectable by the user, andthe second communication icon is made unselectable by the user. Assumingthat a range of brightness values is 0 to 256, in one example, thepredetermined brightness value for the first communication icon is setto 200, and the brightness value for the second communication icon isset to 100. As long as two communications icons can be visuallydistinguished from each other, the brightness value for the secondcommunication icon does not have to be set half of the brightness valuefor the first communication icon. Further, in alternative to thecommunication icons that are selectable and unselectable, informationthat is selectable or not selectable may be displayed in any other form,such as in the form of text data or a thumbnail image. Alternatively,such information including the icons, text data, or thumbnail images,may be displayed differently in terms of color, addition or deletion ofa borderline or an under line, or font type.

The memory 2000 further stores service use information indicatingwhether the videoconference service can be used. More specifically, theservice use information indicates whether the videoconference servicecan be used by the second communication application B10, while the firstcommunication application A11 is being active at the electronicwhiteboard 2 c. The service use information may be registered, forexample, by an administrator of the electronic whiteboard 2 or anadministrator of the communication applications (A11, B10).

Referring to FIG. 12, the combiner 22Y of the electronic whiteboard 2 cobtains the service use information from the memory 2000, at any desiredtime such as when the client 80 is activated under control of the secondcommunication application B10 (S71-1).

The combiner 22Y of the electronic whiteboard 2 c determines whether theservice use information, which is obtained at S71-1, indicates that thevideoconference service can be used by the second communicationapplication B10 on the electronic whiteboard 2 c (S71-2). When theservice use information indicates that the videoconference servicecannot be used (“NO” at S71-2), the combiner 22Y generates image data ofa UI image, without using any of the communication icons stored in thepage data memory 2000 (S71-7). Accordingly, the UI image without anycommunication icon is displayed on the display 3 c. Since nocommunication icon is displayed, the electronic whiteboard 2 c does notaccept any user request for starting communication using thevideoconference service.

When the service use information indicates that the videoconferenceservice can be used (“YES” at S71-2), the communication controller 21 ofthe client 20 obtains state information indicating a connection state ofthe client 80 (S71-3). For example, the communication controller 21 mayrequest the communication controller 81 for state information. Inanother example, the communication controller 81 may send the stateinformation to the communication controller 21 every time the stateinformation is updated.

The combiner 22Y of the electronic whiteboard 2 c determines whether thestate information obtained at S71-3 is online state (S71-4). In thisexample, the online state is a state in which the client 80 logs in themanagement system 50. The online state is mainly classified into thestate “online” indicating that the client 80 logs in the managementsystem 50, and the state “online (communicating)” indicating that theclient 80 logs in the management system 50 and is available forcommunication. When the obtained state information indicates online(“YES” at S71-4), the combiner 22Y reads image data of the firstcommunication icon with a predetermined brightness value, from the pagedata memory 200, to generate image data of a UI image including thefirst communication icon (S71-5). The combiner 22Y controls the display3 c to display the UI image based on the generated image data. Forexample, as illustrated in FIG. 13A, the display 3 c displays a UI image141 a including the first communication icon 141 i having apredetermined brightness value (that is, the brighter icon), which isselectable by the user.

When the obtained state information does not indicate online (“NO” atS71-4), the combiner 22Y reads image data of the second communicationicon of a UI image including the second communication icon, from thepage data memory 200, to generate image data of a UI image including thesecond communication icon (S71-6). For example, as illustrated in FIG.13B, the display 3 c displays a UI image 141 a including the secondcommunication icon 142 i having half the predetermined brightness value(that is, the less bright icon), which is unselectable by the user.

S71-3 to S71-8 are repeated every predetermined time while the client 20is being active (“YES” at S71-8). Accordingly, every time the stateinformation of the client 80 changes, the brightness value of the iconbeing displayed on the UI image changes to reflect the updatedcommunication state of the client 80.

Referring to FIG. 14, operation of starting communication between theelectronic whiteboard 2 c and the PC 8, through the management system50, is described according to an embodiment.

In response to detection of user selection of the communication icon 141i at the drawing controller 22, the electronic whiteboard 2 c displays acommunication menu (S81). FIG. 15 is a flowchart illustrating operationof generating a communication menu to be displayed at S81, performed bythe electronic whiteboard 2 c, according to an embodiment.

As described above, the first communication icon 141 i (“brightericon”), with the predetermined brightness value, is made selectable bythe user. The second communication icon 142 i (“less bright icon”), withhalf the brightness value of the first communication icon 141 i, is madeunselectable by the user. Accordingly, even when the secondcommunication icon 142 i, that is, the less bright icon, is touchedwith, for example, the electronic pen 4 or the user's hand H, selectionof the user is not input (“NO” at S81-1). Further, when the firstcommunication icon 141 i is not selected by the user (“NO” at 81-1), thedrawing controller 22 of the electronic whiteboard 2 c ends operation,without further processing.

When the first communication icon 141 i, that is, the brighter icon, isselected by the user (“YES” at S81-1), the drawing controller 22 acceptsa request for displaying a communication menu, and the operationproceeds to S81-2.

At S81-2, the drawing controller 22 accepts a user input of a user IDand a passcode, as authentication information, on the UI image. Thedrawing controller 22 determines whether a pair of the user ID and thepasscode that is input, matches the pair of the user ID and the passcodethat has been transmitted from the client 80 to the management system 50at S22 of FIG. 9 (S81-2).

When the pair of the user ID and the passcode that is input does notmatch the pair of the user ID and the passcode that has been used forlogin (“NO” at S81-2), the drawing controller 22 ends operation withoutaccepting the request for communication menu. When the pair of the userID and the passcode that is input matches the pair of the user ID andthe passcode that has been used for login (“YES” at S81-2), theoperation proceeds to S81-3.

At S81-3, the detector 23 detects a connection of the external audiodevice to the electronic whiteboard 2 c, from among input/output devicesto be used for inputting or outputting content data (image data andaudio data) to be shared during the videoconference service. Morespecifically, the detector 23 detects the external audio device, basedon whether the OS 2020 detects the microphone 222 or the speaker 223. Incase one of the microphone 222 and the speaker 223 is incorporated inthe electronic whiteboard 2 c, the detector 23 detects other one of themicrophone 222 and the speaker 223 that is not incorporated.

When the audio device is detected (“YES” at S81-3), the communicationcontroller 21 of the client 20 obtains the state information indicatinga communication state of the client 80 (S81-4). The state informationmay be obtained in a substantially similar manner as described abovereferring to S71-3.

When the obtained state information indicates that the client 80 iscommunicating with a counterpart client 80 (“YES” at S81-4), thecombiner 22Y of the electronic whiteboard 2 c generates, as acommunication menu, image data including communication start informationand communication end information each with the predetermined brightnessvalue (S81-5). The communication start information allows the user tostart communication. The communication end information allows the userto end communication.

In one example, when the client 80 is communicating with the counterpartclient 80 on the PC 8 to transmit or receive image data and audio data,the client 80 has the communication state “online (communicating)”. Insuch case, the communication start information, which is selectable bythe user, is generated to be displayed brighter to indicate that theuser can select to start communication. Further, the communication endinformation, which is selectable by the user, is generated to bedisplayed brighter to indicate that the user can select to endcommunication.

More specifically, as illustrated in FIG. 16A, the combiner 22Ytransmits the generated UI image to the display 3 c, to display the UIimage 141 a, which includes a communication menu 143 with thecommunication start information (“START COMMUNICATION”) and thecommunication end information (“END COMMUNICATION”) each being displayedwith the predetermined brightness value. The electronic whiteboard 2 cis able to receive a user input that selects the communication startinformation, to cause the electronic whiteboard 2 c to startcommunication. Further, the electronic whiteboard 2 c is able to receivea user input that selects the communication end information, to causethe electronic whiteboard 2 c to end communication.

When the obtained state information indicates that the client 80 is notcommunicating with a counterpart client 80 (“NO” at S81-4), the combiner22Y of the electronic whiteboard 2 c generates, as a communication menu,image data including communication start information with thepredetermined brightness value, and communication end information withhalf the predetermined brightness value (S81-6).

In one example, when the client 80 is online but not communicating withany counterpart client 80, the client 80 has the communication state“online”. In such case, the communication start information, which isselectable by the user, is generated to be displayed brighter toindicate that the user can select to start communication. Further, thecommunication end information, which is not selectable by the user, isgenerated to be displayed with a less brightness value to indicate thatthe user is not able to select to end communication.

More specifically, as illustrated in FIG. 16B, the combiner 22Ytransmits the generated UI image to the display 3 c, to display the UIimage 141 a, which includes a communication menu 144 with thecommunication start information (“START COMMUNICATION”) being displayedwith the predetermined brightness value, and the communication endinformation (“END COMMUNICATION”) being displayed with half thepredetermined brightness value. The electronic whiteboard 2 c is able toreceive a user input that selects the communication start information,to cause the electronic whiteboard 2 c to start communication. Further,the electronic whiteboard 2 c does not accept a user input that selectsthe communication end information, as the electronic whiteboard 2 c isnot communicating.

Referring back to S81-3, when the audio device is not detected (“NO” atS81-3), the communication controller 21 of the client 20 obtains stateinformation indicating the communication state of the client 80 (S81-7),in a substantially similar manner as described above referring to S71-3.

When the obtained state information indicates that the client 80 iscommunicating with a counterpart client 80 (“YES” at S81-7), thecombiner 22Y of the electronic whiteboard 2 c generates, as acommunication menu, image data including communication start informationwith half the predetermined brightness value, and communication endinformation with the predetermined brightness value (S81-8).

More specifically, as illustrated in FIG. 16C, the combiner 22Ytransmits the generated UI image to the display 3 c, to display the UIimage 141 a, which includes a communication menu 145 with thecommunication start information (“START COMMUNICATION”) being displayedwith half the predetermined brightness value, and the communication endinformation (“END COMMUNICATION”) being displayed with the predeterminedbrightness value. The electronic whiteboard 2 c does not accept a userinput that selects the communication start information, as theelectronic whiteboard 2 c is not capable of processing audio data as theaudio device is not detected. For example, the audio device may not bedetected due to a failure in the microphone 222. Further, the electronicwhiteboard 2 c is able to accept a user input that selects thecommunication end information, to end communication.

When the obtained state information indicates that the client 80 is notcommunicating with a counterpart client 80 (“NO” at S81-7), the combiner22Y of the electronic whiteboard 2 c generates, as a communication menu,image data including communication start information and communicationend information each with half the predetermined brightness value(S81-9).

More specifically, as illustrated in FIG. 16D, the combiner 22Ytransmits the generated UI image to the display 3 c, to display the UIimage 141 a, which includes a communication menu 146 with thecommunication start information (“START COMMUNICATION”) being displayedwith half the predetermined brightness value, and the communication endinformation (“END COMMUNICATION”) being displayed with half thepredetermined brightness value. The electronic whiteboard 2 c does notaccept a user input that selects the communication start information, asthe electronic whiteboard 2 c is not capable of processing audio data asthe audio device is not detected. Further, the electronic whiteboard 2 cdoes not accept a user input that selects the communication endinformation, as the electronic whiteboard 2 c is not communicating.

In this embodiment, as long as communication start information andcommunication end information can be visually distinguished from eachother, the brightness value for the unselectable communicationinformation does not have to be set half of the brightness value for theselectable communication information. Further, information that isselectable or not selectable may be displayed in any other form, such asin the form of an icon or a thumbnail image. Alternatively, suchinformation including the icons, text data, or thumbnail images, may bedisplayed differently in terms of color, addition or deletion of aborderline or an under line, or font type.

As described above, in any one of the communication menus 143, 144, 145,and 146, the information selectable by the user is displayed with thepredetermined brightness value, and the information unselectable by theuser is displayed with half the predetermined brightness value. With thebrightness value that changes according to whether the information isselectable or not selectable, the user can instantly recognize theinformation that is selectable or unselectable. Further, even when theunselectable information being displayed on the UI image 140 a isselected by the user, the drawing controller 22 of the electronicwhiteboard 2 c does not accept such selection.

Referring back to FIG. 14, the drawing controller 22 of the electronicwhiteboard 2 c accepts a communication start request, in response toselection of the communication start information in the communicationmenu 143 or 144, which is selectable (S82). The drawing controller 22further receives a selection of a communication counterpart. In thisexample, it is assumed that the PC 8 is selected as a communicationcounterpart.

The communication controller 21 of the client 20 of the electronicwhiteboard 2 c transmits a communication start request, to thecommunication controller 81 of the client 80. The communicationcontroller 81 of the electronic whiteboard 2 c transmits thecommunication start request to the management system 50 (S83). Thecommunication start request includes a contact ID of the client 80 ofthe electronic whiteboard 2 c as a starting terminal, and a contact IDof the PC 8 as a counterpart terminal. The management system 50 receivesthe communication start request. The contact ID is any identificationinformation of a communication terminal, such as identificationinformation for identifying the communication terminal, and a useraccount ID for identifying the user operating the communicationterminal.

The management system 50 transmits a communication start request to thePC 8 as the communication counterpart (S84). The communication startrequest includes the contact ID of the client 80 of the electronicwhiteboard 2 c, and the contact ID of the PC 8, which are transmitted atS83. The PC 8 receives the communication start request.

The communication controller 81 of the PC 8 transmits a responseindicating whether to accept the communication start request to themanagement system 50 (S85). The response is generated according to auser input indicating whether to accept the communication start request.In this example, it is assumed that the response indicates to accept thecommunication start request.

The communication controller 51 of the management system 50 receives theresponse indicating to accept the communication start request. Thecommunication controller 51 of the management system 50 transmits theresponse indicating to accept the communication start request, to theelectronic whiteboard 2 c as the starting terminal (S86). Thecommunication controller 81 of the electronic whiteboard 2 c receivesthe response.

The management system 50 controls to establish a communication betweenthe electronic whiteboard 2 c and the PC 8 through the communicationnetwork 9 b, to transmit or receive content data (S87). For example, themanagement system 50 assigns an access right to access content data,respectively, to the electronic whiteboard 2 c and the PC 8. The contentdata may be stored in the management system 50 or outside the managementsystem 50. As the communication is established between the electronicwhiteboard 2 c and the PC 8, the electronic whiteboard 2 c or the PC 8accesses a memory area where the content data is stored, with the accessright, to transmit or receive image data and audio data (S88). Morespecifically, the electronic whiteboard 2 c or the PC 8 transmits imagedata of an image captured at the camera 207 or 812, and audio data ofaudio collected at the microphone 222 or 814, to the communicationcounterpart, through accessing. The electronic whiteboard 2 c or the PC8 receives image data and audio data from the communication counterpart,through accessing. While the image data may be video data, the imagedata may be a still image.

As the communication is established between the electronic whiteboard 2c and the PC 8, the communication controller 81 of the electronicwhiteboard 2 c changes the state information in the memory 2000, fromthe “online” to the “online (communicating)” (S89).

The communication controller 81 of the client 80 in the electronicwhiteboard 2 c receives image data and audio data captured at the PC 8.The electronic whiteboard 2 c outputs the received audio data throughthe speaker 223. The communication controller 81 of the client 80 in theelectronic whiteboard 2 c transmits the received image data to thecommunication controller 21 of the client 20. The communicationcontroller 21 of the client in the electronic whiteboard 2 c stores thereceived image data, as medium data, in the page data memory 220. Thedrawing controller 22 reads out the image data from the page data memory220, to be output as a background image layer 140 d through the display3 c. Accordingly, the image data that is received from the PC 8 isdisplayed on the display 3 c, as the background image.

Still referring to FIG. 14, operation of ending communication isdescribed. As described above, in any one of the communication menus143, 144, 145, and 146, the information selectable by the user isdisplayed with the predetermined brightness value, and the informationunselectable by the user is displayed with half the predeterminedbrightness value.

The drawing controller 22 of the electronic whiteboard 2 c accepts acommunication end request, in response to selection of the communicationend information in the communication menu 143 or 145, which isselectable (S91). The communication controller 21 of the client 20 inthe electronic whiteboard 2 c transmits a communication end request, tothe communication controller 81 of the client 80. The communicationcontroller 81 of the electronic whiteboard 2 c transmits thecommunication end request to the management system 50 (S92). Thecommunication end request includes the contact ID of the client 80 ofthe electronic whiteboard 2 c as a requesting terminal, and the contactID of the PC 8 as a counterpart terminal. The management system 50receives the communication end request.

The communication controller 51 of the management system 50 controls toend the communication between the electronic whiteboard 2 c and the PC 8(S93). As the communication ends, image data and audio data are notexchanged between the communication controller 81 of the electronicwhiteboard 2 c and the communication controller 81 of the PC 8.

As the communication ends between the electronic whiteboard 2 c and thePC 8, the communication controller 81 of the electronic whiteboard 2 cchanges the state information in the memory 2000, from the “online(communicating)” to the “online” (S94).

In one or more of the above-described embodiments, the electronicwhiteboard 2 c transmits audio data of audio input from the externalmicrophone 222, to the PC 8. The electronic whiteboard 2 c receivesaudio data, transmitted from the PC 8, and outputs audio of the receivedaudio data through the external speaker 223. The detector 23 of theelectronic whiteboard 2 c detects connection to the audio device (theexternal microphone 222 and the external speaker 223). When the detector23 detects connection to the audio device, the combiner 22Y of theelectronic whiteboard 2 c generates communication start information witha predetermined brightness value (an example of a first display format).When the detector 23 does not detect connection to the audio device, thecombiner 22Y of the electronic whiteboard 2 c generates communicationstart information with half the predetermined brightness value (anexample of a second display format). Through displaying thecommunication start information, which reflects availability of thecommunication function on the electronic whiteboard 2 c, the user caninstantly recognize that communication can be performed using audio datavia the electronic whiteboard 2 c, in addition to communication based onstroke data, for example. That is, the communication start informationis displayed when the external audio device is provided to theelectronic whiteboard 2 c. Further, through changing display appearanceof the communication start information, the user can instantly recognizewhether the audio device is connected or not to enable start ofcommunication. In one or more of the above-described embodiments, whenthe electronic whiteboard 2 c and the PC 8 (as an example of thecommunication counterpart) are communicating, the combiner 22Y of theelectronic whiteboard 2 c generates communication end information with apredetermined brightness value (an example of a first display format).When the electronic whiteboard 2 c and the PC 8 are not communicating,the combiner 22Y of the electronic whiteboard 2 c generatescommunication end information with half the predetermined brightnessvalue (an example of a second display format). Through displaying thecommunication end information, which reflects a communication state ofthe electronic whiteboard 2 c, the user can instantly recognize whethercommunication is being performed. More specifically, through changingdisplay appearance of the communication end information, the user caninstantly recognize whether a communication end request can be acceptedto end the communication.

In one or more above-described embodiments, the client 20 of theelectronic whiteboard 2 c, under control of the first communicationapplication A11, operates as a client that communicates throughtransmitting stroke data. The client 80 of the electronic whiteboard 2c, under control of the second communication application B10, operatesas a client that communicates through transmitting at least audio data.The combiner 22Y of the electronic whiteboard 2 c controls display ofcommunication start information, as a part of the UT image 141 a to bedisplayed under control of the client 20. Accordingly, the electronicwhiteboard 2 c is able to receive a communication start request, throughthe UI image 141 a being displayed under control of the client 20.

In one or more above-described embodiments, the combiner 22Y of theelectronic whiteboard 2 c generates the UI image 141 a, which includesthe communication icon 141 i or 142 i, which indicates availability ofthe videoconference service to the client 80 on the electronicwhiteboard 2 c. When the client 80 is connected to the management system50 that controls communication of audio data (that is, when the client80 is logged on the management system 50), the combiner 22Y of theelectronic whiteboard 2 c displays the communication icon 141 i thatreflects availability of the videoconference service with apredetermined brightness value (an example of a first display format).When the client 80 is not connected to the management system 50 (thatis, when the client 80 is not logged on the management system 50), thecombiner 22Y of the electronic whiteboard 2 c displays the communicationicon 141 i that reflects availability of the videoconference servicewith half the predetermined brightness value (an example of a seconddisplay format). Through displaying the communication icon 141 i thatreflects availability of videoconference service, the user can instantlyrecognize whether the videoconference service is available at all to theelectronic whiteboard 2 c. Further, through changing display appearanceof the communication icon 141 i, the user can instantly recognizewhether the videoconference service is currently available to theelectronic whiteboard 2 c.

While the display appearance of the communication icon 141 i may bedetermined in a substantially similar manner for the case of thecommunication start information or the communication end information,the display appearance of the communication icon 141 i may be determineddifferently than the case of the communication start information or thecommunication end information. For instance, in alternative to changingthe display appearance in brightness value, the display appearance ofthe communication icon 141 i may be changed in any other way other thanthe change in brightness value.

In one or more above-described embodiments, the drawing controller 22 ofthe electronic whiteboard 2 c receives a user input of a user ID and apasscode, as authentication information. The combiner 22Y of theelectronic whiteboard 2 c determines whether a pair of the user ID andthe passcode of the user input (the input authentication information)matches a pair of a user ID and a passcode that has been used for loginby the client 80 to the management system 50 (the authenticationinformation for login). When the input authentication informationmatches the authentication information for login, the combiner 22Y ofthe electronic whiteboard 2 c controls the display 3 c to display thecommunication start information. When the input authenticationinformation does not match the authentication information for login, thecombiner 22Y of the electronic whiteboard 2 c controls the display 3 cnot to display the communication start information. That is, only whenthe user is authenticated, the electronic whiteboard 2 c accepts acommunication start request through the communication start informationthat is made selectable by the user.

In one or more above-described embodiments, when the communication startinformation with the predetermined brightness value is selected by theuser, the communication controller 81 of the electronic whiteboard 2 ctransmits a communication start request to the management system 50.Even when the communication start information with half thepredetermined brightness value is selected by the user, thecommunication controller 81 of the electronic whiteboard 2 c does nottransmit a communication start request to the management system 50. Whencommunication is not available as the external audio device is notconnected, transmission of a communication start request to themanagement system 50 is prevented.

The communication applications A11, B10, and B11 may each be recorded ina file in a format installable or executable on a computer-readablerecording medium for distribution. Examples of such recording mediuminclude, but not limited to, compact disc-recordable (CD-R), digitalversatile disc (DVD), and blue-ray disc. In addition, a memory storingany one of the above-described control programs, such as a recordingmedium including a CD-ROM or a HDD, may be provided in the form of aprogram product to users within a certain country or outside thatcountry.

Any one of the electronic whiteboard 2, the PC 8, and the managementsystem 50 may be implemented as a single apparatus or a plurality ofapparatuses to which divided portions (functions) are allocated. In theabove-described embodiments, the electronic whiteboards 2 and the PC 8are described as examples of apparatus capable of sharing images,however, any other apparatus can be used to share images. Examples ofsuch apparatuses include, but not limited to, videoconference terminal,projector, digital signage, text sharing device, car navigation system,game machine, personal digital assistant (PDA), and remote diagnosissystem.

The above-described embodiments are illustrative and do not limit thepresent invention. Thus, numerous additional modifications andvariations are possible in light of the above teachings. For example,elements and/or features of different illustrative embodiments may becombined with each other and/or substituted for each other within thescope of the present invention.

For example, while the above-described embodiment illustrates the casewhere the communication start information and the communication endinformation are both displayed, only the communication start informationmay be displayed based on a detection of a connection to the externaldevice.

Each of the functions of the described embodiments may be implemented byone or more processing circuits or circuitry. Processing circuitryincludes a programmed processor, as a processor includes circuitry. Aprocessing circuit also includes devices such as an application specificintegrated circuit (ASIC), digital signal processor (DSP), fieldprogrammable gate array (FPGA), and conventional circuit componentsarranged to perform the recited functions.

1. A communication terminal for communicating with a counterpartcommunication terminal, the communication terminal comprising: circuitryto control a display to display communication start informationreflecting a communication function to be started by the communicationterminal, and detect a connection of an external device to thecommunication terminal to generate a detection result, the externaldevice being configured to input or output content data to or from thecommunication terminal to enable the communication terminal tocommunicate the content data with the counterpart communication terminalusing the communication function, wherein, when the detection resultindicates that the external device is connected to the communicationterminal, the circuitry controls the display to display thecommunication start information in a first display format, and wherein,when the detection result indicates that the external device is notconnected to the communication terminal, the circuitry controls thedisplay to display the communication start information in a seconddisplay format.
 2. The communication terminal of claim 1, wherein thecircuitry further determines whether the communication terminal iscurrently communicating with the counterpart communication terminal, andcontrols the display to further display communication end informationreflecting an end of a communication currently performed by thecommunication terminal, wherein, when the determination indicates thatthe communication terminal is communicating with the counterpartcommunication terminal, the circuitry controls the display to displaythe communication end information in a first display format, andwherein, when the determination indicates that the communicationterminal is not communicating with the counterpart communicationterminal, the circuitry controls the display to display thecommunication end information in a second display format.
 3. Thecommunication terminal of claim 1, wherein the circuitry operates undercontrol of first client application and second client application, thesecond client application providing the communication function ofcommunicating the content data input or output through the externaldevice, and the first client application providing a function ofprocessing other content data other than the content data input oroutput through the external device, and wherein the circuitry controlsthe display to display, in a display area controlled by the first clientapplication, the communication start information reflecting thecommunication function to be performed by the second client application.4. The communication terminal of claim 3, wherein the circuitry isfurther configured to: control the display to display, in the displayarea controlled by the first client application, service informationreflecting a service to be used by the communication terminal; anddetermine whether the second client application has established acommunication with the communication management system that controlscommunication of the content data to generate a first determinationresult, wherein, when the first determination result indicates that thesecond client application has established the communication with thecommunication management system, the circuitry controls the display todisplay the service information in the first display format, andwherein, when the determination result indicates that the second clientapplication has not established the communication with the communicationmanagement system, the circuitry controls the display to display theservice information in the second display format.
 5. The communicationterminal of claim 4, wherein the circuitry is further configured todetermine whether first authentication information input from a usermatches second authentication information used by the second clientapplication in establishing the communication with the communicationmanagement system to generate a second determination result, wherein,when the second determination result indicates that the firstauthentication information matches the second authenticationinformation, the circuitry controls the display to display thecommunication start information, and wherein, when the seconddetermination result indicates that the first authentication informationdoes not match the second authentication information, the circuitrycontrols the display not to display the communication start information.6. The communication terminal of claim 4, wherein the communicationstart information displayed in the first display format is selectable bya user, and the communication start information displayed in the seconddisplay format is unselectable by the user, and in response to selectionof the communication start information being displayed in the firstdisplay format, the circuitry transmits a communication start request tothe communication management system.
 7. The communication terminal ofclaim 1, wherein the external device is an audio device capable ofprocessing audio data.
 8. The communication terminal of claim 1, whereinthe first display format and the second display format differ inbrightness value.
 9. The communication terminal of claim 3, wherein thecontent data input or output through the external device by the secondclient application is audio data, and the other content data to beprocessed by the first client application is stroke data.
 10. Acommunication system, comprising: the communication terminal of claim 1;and the counterpart communication terminal to communicate the contentdata with the communication terminal through a network.
 11. Acommunication system, comprising: the communication terminal of claim 4;and the communication management system to establish the communicationwith the communication terminal through a network.
 12. A method ofcontrolling a display of a communication terminal, the communicationterminal capable of communicating with a counterpart communicationterminal, the method comprising: displaying, on the display of thecommunication terminal, communication start information reflecting acommunication function to be started by the communication terminal; anddetecting a connection of an external device to the communication, theexternal device being configured to input or output content data to orfrom the communication terminal to enable the communication terminal tocommunicate the content data with the counterpart communication terminalusing the communication function, wherein, when the detecting indicatesthat the external device is connected to the communication terminal, thedisplaying displays the communication start information in a firstdisplay format, and wherein, when the detecting indicates that theexternal device is not connected to the communication terminal, thedisplaying displays the communication start information in a seconddisplay format.
 13. The method of claim 12, further comprising:determining whether the communication terminal is currentlycommunicating with the counterpart communication terminal; anddisplaying, on the display of the communication terminal, displaycommunication end information reflecting an end of a communicationcurrently performed by the communication terminal, wherein, when thedetermining indicates that the communication terminal is communicatingwith the counterpart communication terminal, the displaying displays thecommunication end information in a first display format, and wherein,when the determining indicates that the communication terminal is notcommunicating with the counterpart communication terminal, thedisplaying displays the communication end information in a seconddisplay format.
 14. The method of claim 12, further comprising:activating first client application, the first client applicationproviding a function of processing other content data other than thecontent data input or output through the external device; and activatingsecond client application, the second client application providing thecommunication function of communicating the content data input or outputthrough the external device, wherein the displaying displays, in adisplay area controlled by the first client application, thecommunication start information reflecting the communication function tobe performed by the second client application.
 15. The method of claim14, further comprising: displaying, in the display area controlled bythe first client application, service information reflecting a serviceto be used by the communication terminal; and determining whether thesecond client application has established a communication with thecommunication management system that controls communication of thecontent data to generate a first determination result, wherein, when thefirst determination result indicates that the second client applicationhas established the communication with the communication managementsystem, the displaying displays the service information in the firstdisplay format, and wherein, when the first determination resultindicates that the second client application has not established thecommunication with the communication management system, the displayingdisplays the service information in the second display format.
 16. Themethod of claim 15, further comprising: determining whether firstauthentication information input from a user matches secondauthentication information used by the second client application inestablishing the communication with the communication management systemto generate a second determination result, wherein, when the seconddetermination result indicates that the first authentication informationmatches the second authentication information, the displaying displaysthe communication start information, and wherein, when the seconddetermination result indicates that the first authentication informationdoes not match the second authentication information, the displayingdoes not display the communication start information.
 17. Anon-transitory recording medium which, when executed by one or moreprocessors, cause the processors to perform a method of controlling adisplay of a communication terminal, the communication terminal capableof communicating with a counterpart communication terminal, the methodcomprising: displaying, on the display of the communication terminal,communication start information reflecting a communication function tobe started by the communication terminal; and detecting a connection ofan external device to the communication, the external device beingconfigured to input or output content data to or from the communicationterminal to enable the communication terminal to communicate the contentdata with the counterpart communication terminal using the communicationfunction, wherein, when the detecting indicates that the external deviceis connected to the communication terminal, the displaying displays thecommunication start information in a first display format, and wherein,when the detecting indicates that the external device is not connectedto the communication terminal, the displaying displays the communicationstart information in a second display format.